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Abstract 


We  show  how  the  duality  theorem  of  linear  progranmiing  can  be  used  to 
prove  several  results  on  general  convex  optimization. 
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Let  f,  g, ,...,g,  be  convex  functions  defined  on  a  convex  subset  S  of 
a  vector  space.   Let  T  =  {x^S  |  g .  (x)_<0  lj<i<k}.   We  assume  throughout  that  T 
is  non-empty. 

We  use  linear  programming  theory  to  explore  the  relationship  between 
the  problem  of  minimizing  f(x)  x€T  and  the  Lagrange  dual  problem  of  minimizing 
f(x)  +  EX.g.(x)  x^s  for  suitable  X.>^0.   This  is  motivated  by  the  work  of  Duffin 
[1,  2,  3].   The  main  tool  we  shall  need  is  a  version  of  the  duality  theorem 
of  linear  programming  [5,  theorems  1.1.9  and  1.7.13]. 

Lemma:  Let  A  =  {x|Bx2^b  and  Cx=c}.  If  A  is  empty,  there  are  IJ>0  and  V 
such  that  U  B+V  C=0  and  Ub+Vc>0.  If  A  is  non-empty  and  every  x€A  satisfies 
dx>e,  then  there  are  U>0  and  V  such  that  U  B+V  C=d  and  Ub+Vc>_e. 

We  will  not  require  separating  hyperplane  theorems  or  results  from 
semi- infinite  programming. 

Theorem  1;   f (x)^L  for  every  x€T  if  and  only  if  for  every  finite  FCS 
there  are  X, ,...,A,  such  that  f (x)+ZX.g. (x)^L  for  every  x^F. 

Proof:   The  "if"  part  is  immediate.   For  the  "only  if"  part  it  suffices 

to  prove  the  result  for  those  finite  F  which  contain  members  of  T.   Let 

F={y^,...,y  }  y,ST.   For  such  F  the  system  of  equations  and  inequalities  in 

unknowns  9 ,,...,  6,, 
1      N 

(°)      E  e.  =  i 
1   ^ 

N 

E  e.g.(y.)^0   llJlk 

e.  >  0 

X   — 
has  the  solution  e,=l.   By  convexity,  if  9 .  is  a  solution  to  (D) ,  ES.y  ST. 
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Since  we  are  assuming  f(x)>L  for  x€T,  every  solution  to  (D)  must  satisfy 
Ze  f(y.)^L.   By  the  lemma  there  are  X,>^0  and  y  such  that 


Y  +   S  X  (-g  (y  ))  <  f(y.)    l<i<N 
j=l  ^   ^ 


Y  >  L 


so  f(yj)  +   ^      X.g.(y.)>L  for  every  y.SF.   Q.E.D. 


(y  )  +   E  X  g  (y  )2iL  for  every  y.SF. 


Theorem  1  may  be  used  to  prove  many  of  the  standard  results  on  convex 
optimization.   As  an  example  we  prove  the  Kuhn-Tucker  theorem. 

Corollary;   Suppose  f(,x)>L   for  x^T  and  that  there  is  a  y  for  which 
g.(y)<0  l£i£k.   Then  there  are  X  ,,..,X,  such  that  f(x)+i:X.g  (x)>^L  for  x^S. 

Proof:   Let  &  =   max  {g.(y)}.   For  x^S  let  A^={  (X^, .  . .  ,X|^)  |X^>^0, 

f(x)+ZX.g.(x)>L,  and  -6(5;X.)<^f  (y)-L}.   For  each  x^S,  A  is  compact.   If  HCS 
1  i  X  X 

is  finite  we  may  use  theorem  1  with  F=Hu{y}  to  show  f^  A  is  non-empty. 
Therefore,   P„  A  is  non-empty,  so  suitable  Lagrange  multipliers  exist.   Q.E.D. 

Arguments  of  this  kind  can  also  be  used  to  give  information  about  when 
"duality  gaps"  occur. 

Theorem  2.   There  are  no  X.  such  that  f (x)+EX .g. (x)>L  for  every  xSS  if 

and  only  if,  for  every  N>0,  there  is  an  x^S  such  that  f(x)<L-N(max  g.  (x)). 

l<i<^k 

Proof:   Take  any  X.>^0  and  suppose  x  exists  with  the  desired  property  for 
N=2:X..   Then  f  (x)+i:X  .g.  (x)j<f  (x)  +  (max  g.(x))  (ZX.)<L.   So  no  suitable  X^  exist. 

Conversely,  suppose  there  are  no  suitable  X..   For  any  N>0  and  any 
F={y,  , .  .  .  ,y-^}CS  consider  the  linear  system  in  unknowns  X  ,...,X, 


f(y^)  +  E  X  g  (y^)>L   y^SF 
3_  J  J 


(F;N)       IX^  <  N 


X.  >  0 
X  — 


If,  for  some  N,  (F;N)  had  a  solution  for  every  finite  FCS,  a  compactness 
argument  similar  to  that  in  the  corollary  to  Theorem  1  would  yield  suitable 
multipliers  X,.   Since  we  are  assinning  such  X.  do  not  exist,  it  must  be  that 
for  every  N>0  there  is  an  F  such  that  (F;N)  has  no  solution.   By  the  lemma, 
if  (F;N)  has  no  solution,  there  are  8-,  ,  •  •  •  .  9j^  2l  0  ^^^  T^O  such  that 

M 

2    Q±sA7J  -  y±o,      l<.j<k 
i=l  ^ 

M 

and  Z9.(L-f(y.))  +y(-N)>0.   By  scaling,  we  may  assume  E  6.=!,  so  that 
IX  3^  1 

**  1 


S  9,g.(y.)lY<N(L-Ee  f(y  )). 
i=l   -■ 

If  we  take  x=E9.y.  y>_g.(x),    l<j<.k  and  f(x)<L-NY  follow  by  convexity  of 

f  and  g..   Q.E.D. 
J 

Corollary :   (Compare  [1],  cor.  5;  [2],  thm.  3):   Let  h(e)=inf {f (x) ] g^ (x)<e, 
l<j_<k.   If  there  are  6>0,  L  such  that  h(x)>L  for  0^x<&,    then  there  are 
X-|,...,Xv^  such  that  f (x)+SX^g^(x)2L  xSS. 

Proof:   If  there  is  an  x  for  which  g  (x)<0  the  existence  of  suitable  X_^ 

it  , 

follows  from  the  Kuhn- Tucker  theorem,   so  we  assume  this  is  not  the  case. 


h  is  a  convex  monotone  function  which,  on  our  assumptions,  is  defined  only 
for  non-negative  arguments.   For  e>6,  h(e)-L>h(e)-h(0)>G(-) (h(6)-h(0)) .   Hem 
for  xGS,  f(x)-L>(max  g.(x))(min  0,  ^(h(5)-h(0)) ) .  (Note  that  our  assumptions 


We  use  the  Kuhn- Tucker  theorem  for  brevity.   The  result  could  be  proved 
from  Theorem  2  alone. 


imply  max  g.  (x)>^0.)  Since  the  condition  given  by  Theorem  2  fails  for 
N=inax  (0,  -^  (h(0)-h(6)),  suitable  X^   exist.)   Q.E.D. 

Finally,  we  use  a  variation  of  these  techniques  to  strengthen  a  recent 
result  of  Duff in  and  Jeroslow  [4]. 

Theorem  3:   Let  S=r'^.   Assume  that  for  X.>0,  f  (x)+EX.g.  (x)>L,  x6S.   Then 

there  are  affine  functions  h. (x)=a.x+b . (a.SR  ,  b.€R)  such  that  h.(x)<g^(x) 

11   1^1   'i'  i^  ''—f'l^ 

and  f  (x)+EX  h  (x)2:L,  xSS. 

Proof:   For  ySS,  let  T  ={(h^,  - . .  ,h^)  |h^  affine,  h(y)_<g(y),  and 
f(y)+EX^h^(y)2L}. 

We  identify  each  function  h.(x)=a.x+b.  with  the  ordered  pair  (a.,b.)Ql 
Thus,  T  is  a  closed  subset  of  R      .  We  first  show  that,  for  any  finite  FCS 
Q  T  is  non-empty.   A  member  of  yQ„  T  would  be  a  solution  to  the  linear 
inequality  system  in  unknowns  a^,...,a,  ;  b,,...,b,  . 

a^y+b^j<g^(y)     llilk;   y€F 

k 
(E)        Z  X  (a  y+b  )>M-f (y)    y€F 
i=l 

By  the  lemma,  (E)  has  no  solution  only  if  there  are  scalars  W,  ,  V  >^0  such  that 

(i)     ^  W  y  =  X   Z   V  y,   l<i<k 
y€F  ^  y€F  ^ 


(ii)     Z  W.   =  X.   Z  V  ,   llilk 
y€F  ^^     ^  y€F  ^ 


(iii)     Z  W, ,g, (y)<   Z  V  (M-f(y)). 
l<±<} 
y€F 


lj<i<^k  ^^  ^      y€F  ^ 


If  there  were  W,V  satisfying  (i)-(xii)  we  could  set  V'*=V  /zV  and 

y  y  y 

W:.=W.  /X.ZV  so  that  (i)-(iii)  would  be  satisfied  and,  by  (ii),  ZV'=  Z  WT  =1, 

ly  ly  1  y  >  j'  ^  ^'  ^  y    ly  » 

l_^i^k.   Condition  (i)  becomes  ZW'  y=ZV'y.   Condition  (iii)  becomes  Z  V'f(y)  + 

^y     y  yGF  y 

ZX  (  Z  W;:  g.(y))<M.   If  z=ZW:  y=ZV'y  this  implies,  by  convexity,  f  (z)+ZX  .gIz)<M 
1  ^j.  ly  X  ly    y  i  i 


which  contradicts  our  assumption  about  the  A..  Therefore  (E)  has  solutions 

for  every  finite  F. 

To  complete  the  proof  we  must  show  0„   T^  is  non-empty.   Let  e.=jth  unit 

vector.  We  show  that  if  F  contains  +e.  l<j<n  and  the  zero  vector,  then  _P-  T 

is  bounded.   Since  each  T  is  closed,  compactness  yields  the  desired  result. 

For  llifk  we  must  have  h.  ($)=b.£g.  (0) ,  f  (0)+I:A  .b  .^L,  h.(e.)=(j:h  component  of  a.) 

+  b,<g.(e.),  and  h. (-e.)<g. (-e. ) .   Since  all  the  X.  are  positive  this  implies 
1-1  J        1   J  -  1   J  1 

bounds  on  a.,  b..   Q.E.D. 
1   i   ^ 

Our  proof  of  Theorem  3  works  for  any  convex  SCR  which  includes  +e.  and 
0.   By  suitable  translations,  this  implies  the  results  for  any  fully  dimen- 
sional convex  SCR  .   Further  modifications  yield  the  result  for  arbitrary 
convex  SCR  . 

I  vould  like  to  thank  Richard  Duff in  and  Robert  Jeroslow  for  their 
encouragement,  anf  for  supplying  me  "sneak  previews"  of  [2]  and  [4]. 
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